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Background / Context: 

Description of prior research and its intellectual context. 

Single-case research (SCR) methods provide a scientifically rigorous approach for documenting 
experimental control and advancing effective practices in education, behavior analysis and 
psychology (Kennedy, 2005; Kratochwill & Levin, 1992; McReynolds & Kearns, 1983; 

Richards, Taylor, Ramasamy & Richards, 1999; Tawney & Cast, 1984; Todman & Dugard, 
2001.) Despite its long history and its potential for providing strong evidence for causal 
inference (Shadish & Rindskopf, 2007), single-case research has not collectively advanced 
evidence-based practice in these areas. Part of the reason can be traced to the fact that no 
acceptable statistical procedures are available for the analysis of single subject designs (SSD) 
and no satisfactory mechanism exists for synthesizing the results from single-case research 
studies. 

Research results today are assessed not only for the extent to which they demonstrate a causal or 
experimental association between interventions and outcomes, but also for the magnitude of the 
effect demonstrated (Cohen, 1988; Hedges, 2007; Hedges, Shymansky & Woodworth, 1989). 

The effect size is increasingly expected to serve as an indicator of the relevance of research 
results (Parker &Hagan-Burke, 2007; Shavelson &Towne, 2002). Single-case research has 
traditionally emphasized the documentation of functional relations (causal connection) through 
visual analysis, but this type of research has not addressed the magnitude of the experimental 
effect. The presumption has been that only large effects would be of importance to an applied 
science of behavior, and that visual analysis would be a valid approach for identifying these large 
effects (Fisher, Kelley, & Lomas, 2003; Parsonson & Baer, 1992). 

The uses of single-case research are now expanding, however, and it is necessary to supplement 
visual analysis with a more formal index of effect size (Parker & Brossart, 2003; Parker et ah, 
2005). The movement to conduct meta-analyses with a body of research to document the basic 
patterns and findings in a field of study has begun to take hold in single subject research. The 
current commitment to documenting “evidence-based practices” (Kratochwill & Stoiber, 2002; 
Odom et ah, 2005; Whitehurst, 2003) further emphasizes the need to move beyond statements 
based on results from any one study, toward documentation of effects supported by an integrated 
set of research results. Without a meaningful effect size index (Hedges, 2007) such syntheses of 
research and documentation of effective practices cannot take place (Horner, Swaminathan, 
Sugai, & Smolkowski, in press). 

Arguably, the most fruitful application of statistical procedures to single-case results may lie 
with developing strategies for measuring effect size. Effect size measures the magnitude of the 
difference in results in standardized units. More than 40 approaches for assessing effect size with 
single-case research have been proposed (Parker & Hagan-Burke, 2007; Kirk, 1996), and more 
recent proposals have become increasingly elegant (Parker, Hagan-Burke & Vannest, 2007). At 
this time, however, each of the available approaches carries important shortcomings (Maggin, 
Swaminathan, Rogers, O’Keefe, & Sugai, 2011). Measuring the size of experimental effects in 
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single-case research will become a necessary element for the broad application of single-case 
results. To this end, an appropriate effect size measure has to be developed (Beretvas & Cheung, 
2008; Hedges, 2007; Shadish & Rindskopf, 2007; Shadish & Rindskopf, & Hedges, 2008). 

In addition to the problem of defining an index of effect size, the major concern raised by single- 
case researchers is the applicability of statistical procedures that fail to take into account the 
realities of single-subject designs. Serial dependence refers to the extent to which the patterns in 
repeated observations are functions of time separation between a series of values. While initially 
researchers (Huitema, 1985) expressed skepticism regarding the existence of serial dependence, 
researchers including Huitema and colleagues (Huitema, 1986; Huitema &McKean, 2007; 
McKnight, Mckean, & Huitema, 2000) have come to realize that serial dependence is an 
importance issue that must be taken into account in the statistical analysis of single-subject 
design and have provided procedures that take into account serial dependencies in the data. 
Shadish et al. (2008) have emphasized this fact in their discussion of the methodological issues 
surrounding the analysis of single-subject designs. 

Purpose / Objective / Research Question / Focus of Study: 

Description of the focus of the research. 

This study is aimed at addressing the criticisms that have been leveled at the currently available 
statistical procedures for analyzing SSD. One of the vexing problems in the analysis of SSD is in 
the assessment of the effect of intervention. Serial dependence notwithstanding, the linear model 
approach that has been advanced involves, in general, the fitting of regression lines (or curves) to 
the set of observations within each phase of the design and comparing the parameters of these 
lines (or curves). In the simplest case of an AB design, this involves fitting a regression lines to 
the observations in phases A and B and comparing the slopes and intercepts. In the event the 
slope is zero in the two phases, the comparison reduces to the comparison of the means or 
levels. If there is a trend but the slope is the same in the two phases, then the adjusted means 
(levels) are compared. A problem arises if the levels and the trends in the two phases are 
different. In this case the intercepts and slopes are compared across the phases. While 
statistically such comparisons are not problematic, the interpretations of treatment effects that 
arise from these comparisons are not intuitive. A “combination” of the slope and intercept 
parameters that yields a single measure of treatment effect could be more meaningful. Such a 
measure of treatment effect yields, in turn, an effect size that can be readily computed and 
interpreted. 

To this end, the study deals with: 

1 . Development of a measure of the effect of intervention which leads to an effect size 
measure 

2. Development of statistical procedures for assessing intervention effects that take into 
account the serial dependence in single-subject designs; 

The proposed statistical procedures are based on linear models, take into account the serial 
dependency in the observations, and are applicable when there is a mean shift as well as trend. In 
addition, an effect size measure that lends itself to meta-analysis is proposed. 
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Setting: 

Description of the research location, s 

(May not be applicable for Methods submissions) 

Not Applicable 

Population / Participants / Subjects: 

Description of the participants in the study: who, how many, key features, or characteristics. 
(May not be applicable for Methods submissions) 

Not Applicable 

Intervention / Program / Practice: 

Description of the intervention, program, or practice, including details of administration and 
duration. 

(May not be applicable for Methods submissions) 

Not Applicable 

Significance / Novelty of study: 

Description of what is missing in previous work and the contribution the study makes. 

As described in the previous section, this study provides a measure of treatment effect in single- 
subject designs, and consequently an effect size measure which could be used effectively in 
meta-analysis of single subject designs (Maggin, Swaminathan, Rogers, O’Keefe, & Sugai, 
1991). In addition, analytical procedures that take into account serial dependency in the 
observations for testing hypotheses regarding the effect of intervention is provided. 

Statistical, Measurement, or Econometric Model: 

Description of the proposed new methods or novel applications of existing methods. 

In the simplest case of an AB design, the linear models fitted to Phases A and B are: 

yt = Pg + Pf ^ + = 

yf = y^(f + Pi t + e, {t = n^+l,n^+ 2,....,n^ + nf} 

If we set pf = = 0 , we obtain a model where there is only a level change between the phases; 

on setting pf = p^ = /?j we obtain the model where there is a shift in the levels adjusted for the 

trend. Following the logic provided in Shadish, Cook, & Campbell (2002) in regards to quasi 
experimental time series designs, we conclude that if the trend lines in the two phases coincide, 
the intervention is not effective. This the effect of the intervention may be defined as the 
difference between the pints on the trend line in Phase B and the points of the Phase A trend line 
that is projected into Phase B. At point any point nA +t in phase B, the difference between the 
trend line in Phase B and the projected trend line from Phase A is 

^ABU) = yB\B(t) - yB\A(t) = iPo -Po) + iPl + 0 

This difference is calculated at all points in Phase B and averaged to yield the treatment effect 
defined as 

A... = ( A* - ) + ( A" - fit . 
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The estimate of the treatment effect measure defined above is obtained by substituting the 
estimates of the regression coefficients. It follows that if A = 0 we can conclude that the 
intervention is not effective. To test the hypothesis A=0 (against the alternative, A 0), the ratio 

D! SE{D) is formed and has an approximate t-distribution with degrees of freedom {N - 5), in 
the presence of first order autoregressive errors. 

In the estimation of the regression coefficients and consequently D, and in particular, the 
standard error of D, the serial dependency in the observations must be taken into account. 
Maximum Likelihood and the Cochran-Orcutt procedures are employed to estimate the 
autocorrelation parameter, the regression coefficients, the error variance, and finally, D. 
Asymptotic as well as conditional standard errors are computed for testing the hypothesis A =0 
and for constructing confidence intervals around A . 

The parameter A is clearly an effect size measure. A standardized effect size measure is obtained 
by dividing by the standard deviation obtained by pooling the error variances in the two phases. 
An approximate distribution of the standardized effect size measure is obtained in this study. 

Usefulness / Applicability of Method: 

Demonstration of the usefulness of the proposed methods using hypothetical or real data. 

The methods described above are applied for the analysis of several published single -subjects 
time series data. Examples include AB, ABAB, and multiple baseline studies. While effect size 
measures for examining the effectiveness of the treatment in adjacent phases are important, an 
overall measure of treatment and effect size across the phases of an ABAB design and multiple 
baseline designs may prove to be useful (Homer, et ah, in press). An average effect size measure 
is proposed in this paper for this purpose. These analyses are compared with the assessment of 
treatment effect obtained through visual analysis. For the data analyzed, statistical analysis and 
visual analysis yield similar results indicating that the statistical procedures outlined here are 
indeed viable and appropriate for the analysis of single-subject designs. 



Conclusions: 

Description of conclusions, recommendations, and limitations based on findings. 

The statistical procedure developed here that includes a definition of treatment effect and 
consequently an effect size provides a viable method for the analysis of single-subject designs. 
Then advantage of the procedure outline here is that a single measure of treatment effect and 
consequently a single effect size measure is available when there is both a level change and trend 
change in the phases. Furthermore, the serial dependency in the observations is taken into 
account in the analysis of the data. 

Single-subject designs are very short time series and this presents a problem in estimating the 
parameters in the autoregressive process assumed for the errors. A Bayesian approach that has 
the potential for overcoming this limitation is described in the paper. The posterior distribution 
of the effect size measure is obtained using the Markov-Chain Monte-Carlo procedure. The 
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posterior distribution provides a non-asymptotic credibility interval for the treatment effect. 
Research on the posterior distribution of the standardized effect size measure is under way. 
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